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Focus on Criterion-Related Validity 

(or Predictive Vaiidity) 


Predictive validity refers to the "power" or 
usefulness of test scores to predict future 
performance. 

Over time, validity evidence will continue to gather, 
either enhancing or contradicting previous findings. 

Establishing predictive validity is particularly useful 
when colleges or universities use standardized test 
scores as part of their admission criteria for 
enrollment or for admittance into a particular program. 


• This is also the responsibility of the test publisher. 
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National SAT Validity Study 


Cross-institutional, longitudinal validity and higher education 
research informing ways to ensure that students are ready for 
and successful in coliege. 

Data supplied by four-year institutions from around the U.S. and 
matched to College Board data. 

Topics studied inciude: 

• Predictive validity of SAT with regard to FYGPA, cumulative GPA, retention (will study 
graduation when those data are available) 

• Understanding discrepant performance on SAT and HSGPA - implications for college 
performance 

• AP participation and performance and related college outcomes 

• Relationship between self-reported and actual HSGPA 

• Characteristics of students who switch from and remain in STEM majors 

• Many more ^ CollejgeBoard 
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Sampling Plan 

(developed in 2006) 


The population of colleges: 726 institutions receiving 
200 or more SAT score reports in 2005. 

The target sample of colleges: stratified target 
sample was 150 Institutions on various 
characteristics (public/private, region, admission 
selectivity, and size) 

Institutions have been recruited via: E-mail invites 
and/or visits from CB staff; Conference Exhibit 
Booths; Print announcements in CB and Association 
for Institutional Research (AIR) publications; etc. 
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Institutional Characteristics (A/=110) 

(entering class of Fall 2007 - 1st Yr) 
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File Submission Takes Place with the 
Admitted Class Evaluation Service (ACES) 


ACES is a free online service that predicts how 
admitted students will perform at a college or university, 
generally (admission validity), and how successful 
students will be in specific courses (placement validity). 


By using ACES to submit the SAT Validity Study file, 
each institution receives a unique admission validity 
study and a returned file with supplementary variables 
from the College Board database (e.g. AP scores, SAT 
Questionnaire responses, etc.) 

www.collegeboard.com/aces 
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National SAT Validity Study Data in House 


/ Fall 2006 Entering Cohort 
1 Year 

• 110 institutions 

• 196,364 students across the US; 
151,316 students had complete data 
(SAT, HGPA, FYGPA) 

2"^ Year 

• 67 returning institutions 

• 109,153 students across the US; 
-74,955 students had complete data 
(SAT, HGPA, FYGPA, SYGPA, 
cumGPA) 

Year 

• Importing in progress 
\ • 60 returning institutions 




Fall 2007 Entering Cohort 
1®* Year 

• 1 10 institutions 

• 216,081 students across the US; 
159,286 students had complete 
data (SAT, HGPA, FYGPA) 

2"^ Year 

• Importing in progress 
94 institutions submitted data 


Fall 2008 Entering Cohort 
1®* Year 

• Importing in progress 

• 130 institutions submitted data 


n 
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Data Included in Files 


For example, a First-Year Data on Fall 2008 Cohort (first-time, first-year 
students that began at institution in fall 2008) would contain students’: 

• Name 
. SSN 

• Date of birth 

• Gender 

• University-assigned student ID 

• Retention to the year ("yes" or "no") 

• First-year GPA 

• Grades in first-year courses 

• Course abbreviations for first-year courses (e.g., ENG 101) 

• Course long names for first-year courses (e.g., Introductory English) 

• Credit hours attempted for each course 

• Semester each course was taken 

• High School GPA (can be supplied by the ACES system or the institution) 

After year of data, we also ask for Major and CIP code 
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Institutional data matched to CB 
records: 


• Test scores (SAT, AP, SAT subject tests, PSAT/NMSQT) 

• SAT Questionnaire responses 

• Gender 

• Race/Ethnicity 

• Self-reported HSGPA 

• High school coursework and activities 

• College plans 

• Annual Survey of Colleges (institutional characteristics) 

• Size 

• Selectivity 

• Control 


Region 
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Cleaning the Data after ACES Processing 


Student Level Checks to Remain in the Study 

• Student earned enough credit to constitute completion of a full academic year 

• Student took the SAT after March 2005 (SAT W score) 

• Student indicated their HSGPA on the SAT Questionnaire (when registering for the SAT) 

• Student had a valid FYGPA 

Institution Level Checks to Remain in the Study 

• Check for institutions with high proportion of zero FYGPA (should some be missing or null?) 

• Grading system makes sense (e.g. an institution submitted a file with no failing grades) 

• Recoding variables for consistency (e.g. fall semester or fall trimester or fall quarter = term 1 
for placement analyses) 
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Validating a Test for a Particular Use 


The most common approach used to validate an admission 
test for educational selection has been through the 
computation of validity coefficients and regression lines. 

Validity coefficients are the computed correlation 
coefficients between predictor variables and a criterion or 
outcome variable(s), which can determine the predictive 
validity of a test. 

A large correlation indicates strong predictive validity of a 
test to the criterion, however, a large correlation by itself 
does not satisfy all facets required of test validity. 
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SAT Validity Study results - snapshot 


Admission Validity Study 


SAMPLE (2007 entering cohort) 

110 colleges participating in Validity 
Study (N = 216,081) 

• Schools provided first year performance data 
for Fall 2007 cohort through the Admitted 
Class Evaluation Service™ (ACES ™) portal 


Restrict sample to students who 
completed the new SAT, submitted 
self reported HSGPA, and had a valid 
FYGPA (N=1 59,286) 
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Admission Vaiidity Resuits (1 of 2) 


• SAT Writing has the highest correlation with 
FYGPA among the three individual SAT sections 
{Adj. r= 0.53). 

. SAT CR {Adj. r= 0.50); SAT M {Adj. r= 0.49) 


• As expected, the best combination of predictors of 
FYGPA is HSGPA and SAT scores (Adj. r=0.64), 
reinforcing the recommendation that colleges use 
both HSGPA and SAT scores to make the best 
predictions of student success. 
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Admission Vaiidity Resuits (2 of 2) 


• The adjusted correlation of HSGPA and FYGPA is 0.56, 
which is the same as the multiple correlation of the SAT 
(CR, M, and W combined) with FYGPA (Adj. r= 0.56). 

• The increment in predictive validity attributable to the SAT 
when HSGPA is taken into account is 0.08. 

• The increment in validity attributable to the Writing section 
over and above the CR and M sections is 0.02. When 
HSGPA is also considered, the increment in validity 
attributable to the Writing section is 0.01 . 
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FYGPA 


Another way to think of a correlation of 0.53 
Mean FYGPA by SAT Score Band 
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FYGPA 


Another View of Incremental Validity 

Mean FYGPA by SAT Score Band, Controlling for HSGPA 




Differential Validity and Prediction 

Differential Validity: refers to a finding where the computed 
validity coefficients are significantly different for different 
groups of examinees. (A test can be predictive for all groups 
but to different degrees.) 

Differential Prediction: refers to a finding where the best 
prediction equations and/or the standard errors of estimate 
are significantly different for different groups of examinees. 
Differential prediction is therefore the result of varying 
degrees of validity for the variables across examinee 
groups. 

• Underprediction: Performing better in college than was predicted. 

• Overprediction: Performing worse in college than was predicted. 
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Differential Validity 


Similar to previous findings. . . 


Differential Validity Results 



• SAT and HSGPA were more predictive of FYGPA (higher 
correlations) for females versus males, White students versus 
other racial/ethnic groups, and students indicating English as 
best language versus English and Another or Another 
language as their best language. 


• Within subgroups, SAT scores (versus HSGPA) were more 
predictive of FYGPA for females, American Indian or Alaska 
Native, Asian, Black, and “Other” students, as well as those 
indicating their best language to be Another language or 
English and Another. 
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Differential Prediction Results 


Differential Prediction 

• SAT and HSGPA tend to underpredict FYGPA for females; however, 
magnitude is larger for the SAT. 


• SAT and HSGPA tend to overpredict FYGPA for minority students; 
however, magnitude is larger for HSGPA 


• SAT-CR & SAT-W tend to underpredict FYGPA for students whose 
best lang. is not English. SAT-M accurately predicts their FYGPA. 


• SAT & HSGPA both tend to overpredict FYGPA for students whose 
best lang. is English and another language; however, magnitude is 
larger for HSGPA. 


20 



CollegeBoard 


inspiiing mind^ 


SAT and Retention 


This study answers: 

• Is performance on the SAT related to retention? 

• What are the demographic characteristics of 
returners vs. non-returners? 

• Similarly, do retention rates vary by student and 
institutional characteristics? 

• If so, are these differences reduced or eliminated when 
controlling for SAT performance? 
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Sample & Measures 

Sample 

• Analyses based on data collected for the national SAT 
Validity Study 

• The sample included the 147,999 students (106 institutions) that had 
complete data (SAT, HSGPA, retention) 

Measures 

• Institutions provided retention 

• SAT scores (most recent) were obtained from CB records 


• HSGPA was self-reported, obtained from the SAT- 
Questionnaire 
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Analyses & Results 


Comparison of returners (86%) and non-returners 

• By student and institutional characteristics (%) - Of note: 

• % of non-returners that are American-Indian, African- 
American, and Hispanic were slightly higher than for the total 
group. 

• Students from lower SES families made-up a greater 
percentage of the non-returners as compared to the total 
group. 

• 15.4% of the sample attended a selective institution (i.e., 

admits fewer than 50% of applicants); however, this 
percentage varied markedly for returners (16.8%) and non- 
returners (7.2%). rsO 
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Comparison of Returners and Non-returners 

• Mean performance on academic indicators 


Measures 

Returners 

Non-returners 

SAT - CR 

562.5 

526.3 

SAT-M 

580.8 

538.7 

SAT-W 

556.2 

516.8 

HSGPA 

3.6 

3.4 
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•On average, returners had a SAT total score that 
was 97 points higher as compared to non- 
returners. 


Collejg^oard 

inspiiing mind^ 


Analyses & Results 

Retention rates by: 

• Academic characteristics (SAT, HSGPA) 

• Student characteristics (Gender, Race/Ethnicity, Parental 
Income and Education) 

• Academic x Student Characteristics 

• Institutional characteristics (Control, Size, Selectivity) 

♦ Academic x Institutional Characteristics 
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% Returning 


Retention Rates by Academic Characteristics 


Second Year Retention Rates by SAT Score Band 



SAT Score Band 
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% Returning 


Rates by Academic Characteristics 


Second Year Retention Rates by HSGPA 
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HSGPA 
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Even within HSGPA categories, SAT 
provided additionai information... 


Retention Rates by HSGPA Category by SAT Score Band 
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Retention Rates by Student Characteristics 

• Gender: 86.3% - females; 85.7% - males 

• Race/ethnicity: ranged from 89.3% for Asian 
students to 78.6% for American Indian students 

• SES: As parental income and education increased, 
retention rates increased from 82% to 87% 

• Differences in retention rates by student 
characteristics are minimized and, in some 
instances, eliminated when controlling for SAT 
scores. 
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Retention Rates by Institutional Characteristics 



Retention Rate (%) 
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Retention Rates by Institutional Characteristics 


Variable 


n 

Retention 
Mean SD 

Overall 


147,999 

86.0 

34.7 


Private 

45,761 

88.9 

31.4 

Control 

Public 

102,238 

84.7 

36.0 


Small 

6,430 

82.1 

38.3 


Medium 

30,110 

86.1 

34.6 


Large 

41,851 

84.9 

35.8 

Size 

Very large 

69,608 

87.0 

33.6 


Under 50% 

22,848 

93.5 

24.7 


50% to 75% 

84,784 

85.7 

35.1 

Selectivity 

Over 75% 

40,367 

82.5 

38.0 


u 
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Summary 


• Performance on the SAT is related to college 
retention 

• Retention rates by SAT score bands vary substantially 
with only 63.8% percent of low performers returning 
versus 95.5% of high performers 

• This is true even after controlling for HSGPA 

• Retention rates do vary by student and institutional 
characteristics 

• This is partly attributable to differences in the academic 
achievement level 


32 



CollegeBoard 


inspiiing mind^ 


Understanding Students with 
Discrepant SAT scores and HSGPA 


•This study examines: 

•The frequency of students with discrepant HSGPA 
and SAT performance (difference ^ 1 SD) 

•Whether certain students are disproportionately more 
likely to exhibit discrepant performance 

•Among those with discrepant performance, which 
measure is more indicative of college performance 
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Distribution of Students 
by SAT-HSGPA Discrepant Groups 


Discrepant Groups 

Frequency 

Percent 

Higher HSGPA 

26,094 

17.4 

Nondiscrepant 

98,025 

65.2 

Higher SAT 

26,258 

17.5 

Total 

150,377 

100.0 
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Performance on Academic Measures 
by SAT-HSGPA Discrepant Groups 

Higher HSGPA Nondiscrepant Higher SAT 


Variable Mean SD Mean SD Mean SD 


SAT total 

1468 

177 

1705 

231 

1871 

247 

SAT-CR 

480 

71 

564 

87 

626 

93 

SAT-M 

509 

79 

583 

91 

632 

93 

SAT-W 

479 

70 

558 

87 

614 

94 

HSGPA 

3.94 

0.31 

3.63 

0.45 

3.16 

0.54 

HS Rigor 

2.24 

1.85 

2.98 

2.07 

3.30 

2.11 

FYGPA 

2.91 

0.69 

3.01 

0.69 

2.90 

0.76 

detention 

86.8 

33.8 

88.3 

32.2 

OR 

OD.*F^ ingpiimg-Tikindif 


FYGPA 


FYGPA of SAT-HSGPA Discrepant Groups 

by HSGPA 


FYGPA of SAT-HSGPA Discrepant Groups by HSGPA 
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Higher HSGPA^^ Nondiscrepant* Higher SAT 
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Error 


0.30 

0.20 

0.10 

0.00 

- 0.10 

- 0.20 

- 0.30 


Average Overprediction (-) and Underprediction (+) of FYGPA for HSGPA and 

SAT by SAT-HSGPA Discrepant Groups 



Predictor(s) 


Higher HSGPA ■ Nondiscrepant ■ Higher SAT 


Summary 


• Over one-third of students exhibited discrepant 
performance. 

• Using only HSGPA for admission under-predicted 
college performance for those students who 
performed significantly higher on the SAT as 
compared to HSGPA. 

• Results underscore the utility of using both 
HSGPA and test scores for admission decisions. 
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Admitted Class Evaluation Service (ACES) 


• The Admitted Class Evaluation Service (ACES) 
is a free online service that predicts how admitted 
students will perform at a college or university 
generally, and how successful students will be in 
specific classes. 

http://professionals.colleqeboard.com/hiqher- 

ed/validitv/aces 
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About ACES 


• ACES offers two models of validity studies: 

• Admission 

• Predictive 

• Placement 

• Predictive 

• Concurrent 
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Admission Vaiidity Studies 


• The primary purpose of an admission validity study is to 
validate measures used in admission decisions. 

• Can determine how well admission criteria work alone 
and in combination with other predictors, and the most 
effective weighting for the predictors. 

• Success (the criterion) may be measured by college GPA 

• Relevant predictors may be 

• SAT scores - Critical Reading, Math, or Writing 

• High school GPA, or Class Rank 

• Interview scores, and 

• Other information 
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Requesting an Admission Vaiidity Study 


• A minimum of 75 student records is required for an 
admission study. 

• You may specify up to 5 additional predictors - either from 
ACES-supplied data or from your institution (provided that 
75+ students in your sample have that additional variable). 

• ACES automatically breaks down the results of your study 
on the basis of gender, race/ethnicity, and first language 
spoken (provided that there are 75+ students in the 
sample in at least 2 levels of the subgroup) 

• You may also specify 2 additional subgroups - either ACES- 
supplied (e.g. degree-level goal, ability rating in math), from your 
data (e.g. resident versus commuter), or a combination. 
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Overview of ACES Process 


• The institutional contact/submitter will: 

1 . Click link on ACES web site for a new ACES study request: 
https://cbweb1s.collegeboard.org/aces/htnnl/newrvs.htnnl 

2. Enter contact info (name, email, position, institution, etc.) 

3 . Design study (choose predictors, subgroups, etc.) 

4 . Receive automatically e-mailed user account, password, and request 
number from ACES 

5 . Login to submit data at this site: 
https://cbweb1s.collegeboard.org/aces/html/submit1.html 

6. Record all variable locations, indicate value labels, etc. 

7 . Upload data file(s) 

8. ACES reports are returned to institutions 25 - 35 business days after the 
receipt of clean data. 
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ACES Web Site - Requesting a Study 


( CollegeBoard 
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Education Policy Si 
Advocacy 


College 

Guidance 


Membership Testing 
Educators - Information Sc Tools For Teachers. Counselors, Higher Education Faculh^ and 

Higher Ed Services 


ACES 


Recruitment 3c Admesions 


Financial AkJ 




MORE INFORMATION ABOUT ACES 




FactShe&t(.pdf/124K} 

Request an Admission Validity Study 
Request a PlacemerfrValidity Study 


Advising 3c Placerment 


Validity 

ACES™ 

Ad missio n Va IkJ ity Study + 

Placement Validity Study + 

Validity Handbook + 


ACES offers you free, easy-to-read validity studies 

ACES offers tv.ro types of validity studies— admission and placement. These 
studies identify the optimum combination of measures to predicta students future 
performance at your institution. ACES studies evaluate the differences for predicting 
the success of specific student groups and document the probabiiity of error. Each 
AGES report features: 

* In-depth analysis offindings 

* General background information to help you examine the study in greater 
detaii 

* Interpretive text highlighting key findings 



I K-12 Teacheir 


Fact Sheet C.pdf/124K} 

Request an Admission Validity Study 
Request a Placement Validity Study 

CONTACT 
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* Colorful presentation and graphics 

AGES admission validity studies typically use high school grade point average 
[HSGPA) or high school rank, along with SAT Reasoning Test™ scores, to establish 
the best combination of variables to predict student performance at your institution. 
You choose additional variables based on what you believe to be important 
contributors for predicting the academic success of your students. Examples of 
additional predictors are: 

* SAT SubjectTests™ scores 


Admitted Class Evaluation Service™ 
(ACES™) 

Research and Develop nrent 
The College Board 
45 Columbus Avenue 
New York, New York 1 0023-S992: 

Phone: (STT) SS5-2237 
FAX: (212} 649^27 

Questions about ACES online: 
a ces©in to . co llegeb o a rd . o rg . 


* Years of study in a particular subject area 
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(contact information) 
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Admitted Glass Evaluation Service 


Admission Validity Study Request 


"Lse this form to submit a first request prior to submitting your institution's data. 

AH ACES Admission \"alidit>' Studies use first-year grade point average as the default criterion . You ha^'e the option of specifting a diflEerent criterion in Step 2 and of customizing other aspects 
of y our ^^alidit>' study in Step 3 . 

IfciT any time you have questions about the request process you may e-mail AC£S staff for assistanc.e. 

Coufact lufot'uiatiou 


(ISrote: Items marked by an " are required) 


Kame of institution: * 
Institution's College ^ 
Board code number: 

Last name 
First name 
M.I. 
Position Title 
E-mail address 
Telephone number 
Street 
Cit>^ 
State 
Zip code 
Department or school 
Secondary" contact 


Desigu Your Report 

Continue to Step 2 \-s"here you can specift" your criterion and predictors. 


Please enter this exactly as it is to appear in your final report. 
Institution code look-up: click here fPop-ups must be enabled to v;en’ this link) 


Clear Form 


45 


Colleg^oard 

inspiiing mind^ 



study Design 

Specify Criterion; Specify Predictors - HS measure 


^Specify a Criteriou 

You may use the default criterion, first-year grade point average, or ripe in your o’h^n. Remember to ripe all information exactly as you would like it to appear iu your final report. 


I ijmit of 20 characters) 

Specify Predictors 

All ACES Admission Validiri^ Studies use high school grade point average (GPA) or class rank. SAT Reasoning Test scores and SAT Subject Test scores (optional) as predictors. Please 
specif}^ below your particular preferences for how these predictors should be used in your study. 

Specify GPA or Clan Rafik 


All Admission Validiri^ Studies use eitiier high school GPA or high school class rank as predictors. Please mdicate below which predictor you would prefer. You must also specift' if you will be 
pro\iding this information or if you want to use ACES -supplied data. 


T\~pe of high school data ” 


Source of data ” 


C HSGPA 
C HSrank 


C from data supplied by your mstitution 
C from ACES -supplied data 
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Specify Predictors - SAT Scores 




SAT Reasoning Test: 
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Individual Predictors 

Multiple Predictor Sets 

Single Scores 


Critical Reading 

Critical Reading, Math, Writing 

r Math 

C [Critical Reading - Math], Writing 

C Writing 

C Critical Reading, Writing 

Composite Scores 

C Critical Reading, Math 

(sum of scores) 

C Math, Writing 

C [Critical Reading -Math] 


C [Critical Reading -Writing] 


C [Math - Writing] 


C [Critical Reading -Math -Writing] 


SpecifH- highest or most recent score(s) ” 

C Use the highest score(s) 


C Use the most recent score(s) 
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Specify Predictors - SAT Subject Tests 


score(s) on specific SAT Subject Test(s) 

- OR- 


Using liigliest or average SAT Subject 
Tests 

Select fu st SAT Subject Test predictor 


■ 

1 

Select fu st SAT Subject Test predictor 


EnglLsh j 

L^mgu^lges 

1 

O Literature 

C Chinese with Listening 


C French 

Histoiy & Social Sciences 

C French with Listening 

r U. S. History 

C German 

O World Histoiy 

C German with Listening 


C Modem Hebrew 

Mathematics 

C Italian 

Mathematics Level 1 

C Japanese with Listening 

^ Mathematics Level 2 

C Korean with Listening 


C Latin 


C Spanish 

Science 

C Spanish with Listening 

O Biology 


Chemistry 


^ Physics 



- OR- 


C Nuii-Laii^uti^d 3 AT SubjduL TdbL 

Highest of AJl SAT Subject Tests 

Average of Tv^o Highest Non-Language 
SAT Subject Tests 

C Average of Two Highest SAT Subject Tests 
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Specify Additional Predictors 
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}H Specify Adtlidoii^l Piedictors 


You may also specify as many as five additional pieces of information to be used as predictors. These can be drawn either fi-om ACES -supplied data or fi-om 
information supplied and defined by your mstitution, providing that at least 75 students m your sample have the additional variable (s) in their records. Variables 
to include would be those that are used m making admission decisions at your institution. 


Using ACES- supplied data 


OR specify your own: 


r Years of study in arts and music 
r Years of study in EngHsh 
r Years of study in foreign and classical languages 
r Years of study in mathematics 
r Years of study in natural sciences 
r Years of study in social sciences and history 
r Number of different SAT Subject Tests 
r Number of different AP exams 
r Honors 

r Number of activities 


Using yom’ data 



1 



1 



1 



1 
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Examine Subgroups; Include Coursework 


Admission Validity Study Request (continued - 3 of 3) 

Specify Ad<Urioii»l Subgioups 

AJl ACES studies break down your results on the basis of gender, etlmiah^ and native language spoken whenever your sample includes 75 
or more students for at least two levels of a subgroup (e.g., 75+ males and 75+ females). You may also specify up to two additional subgroups 
below, again, using either ACES -supplied data , your own data , or a combination (i.e., one subgroup from each). 


Select Additional Snbgi onp 1 


Select Additional Snbgronp 2 


C Degree-level goal 
C Ability rating in math 
C Ability rating in science 
C Ability rating in writing 
r Other I 


C Degree-level goal 
C Ability rating in math 
C Ability rating in science 
C Ability rating in writing 
r Other I 


Decide Mieflier io Accoimi for Comse-Takiiig Behavior 


By submitting college course -level data about your students for analysis, you may get more significant results. Submitting course -level data 
requires that you provide the following information for each student: for every course taken you will need the name of the course (e.g., 
"ENGlOl"), the grade received, and the number of credits earned. 


Would you like results for all Hispanic ethnic groups to be analyzed and reported together as one group? (Note: In order to analyze and report 
these groups separately, you must have no less than 75 students of one or more of these three Hispanic subgroups: Mexican or Mexican 
American; Puerto Rican; or Latin American, South American, Central American, or other Hispanic or Latino.) 

Yes r No 
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ACES Data Submission 


• Choose from a variety of common formats including: 

• Excel 

• Access 

• SPSS 

• SAS 

• ASCII delimited format 

• Upload the file directly from a PC to ACES 

• ACES automatically encrypts the data during transmission 
to protect confidentiality 
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Submitting Data (cont.) - Course grades, Number of fiies 


Submitting Your Data (continued) 

If at any tim&you have quQSrtions about ths data SrUhmiSr^ion procoSrS. you may' Q-mail A CES staff for assisrianco. 

Course Grade luformadou 

You indicated on your request that you ’^^'ould be submitting college course -level data. Please ans\^'er the questions below speciftiug how you ’^^iU be submitting these data. 

’V^’iU grades appear in numeric or letter form? 

f* letter grades 
C numeric grades 

\Miat is ±e range of possible numeric grades, or numeric equh'alents of alphabetic grades, at your school'^ 

Lowest^ I Highest'^ | 

\Miat is the maximum number of courses per student for ’\\'hich grades ’h^iU be submitted'^ 

|- Please Select - 

Data liLe luformadou 

Ylien providing course grade data, you have the option of submitting a single file T.^ith all student data or submitting two separate files, one \^ith admission-related data, the other ’^^ith course grade 
data. Ylien submitting separate files, certain requii'ements and restrictions and file formatting rules apply, ’H^'hich you may want to re\iew. 

In how many data files (and if t\\'o, in what format) you be submitting data for this study request'^ 

submitting one file (horizontal format) 

C submitting two files (both in horizontal format) 

C submitting two files (^^ith course grade file in ^'ertical format) 


Continue | Clear All Values | 



Submitting Data (cont.) - Labeling mto 


Submitting Your Data (continued) 


If at any time you have questions about the data subtni^sio?^ process you mai' e-mail ACES staff for assistance. 

About Your Data lUe 


Please select the year that students represented m your data entered college|2008 ^ (required) 

Please indicate ’\^'hat you ’h^'ould like to call the data file you wiS. be submitting. This should be a simple descripth'e label, e.g., "Entering Class of Fall 2008." 


File label: f 


(required) 


Please specific how your student name is formatted: (required) 
f* In separate fields: first middle last (any order; e.g., Public 


Jolm 


) (recommended) 


C In a single field: last, first, middle (e.g., 
C In a single field: first, middle, last (e.g., 


Public, John Q. ) 


John Q. Public ) 


Specific your file t>pe belo\^^ (required) 

YEcrosofl: EXCEL Spreadsheet C Tab -Delimited ASCII C Fixed Length ASCII 
C Xlicrosoft ACCESS Database C SPSS Portable File 
C Comma-Delimited ASCII (CS\0 O SAS Transport File 


Continue Clear all values 
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Submitting Data (cont.) - variable locations/values 


Delimited (tab, comma) ASCII or XLS (Excel) Layout Table 



Label 

Excel CoLumu 
No. or ASCn 
Var. Fosidoo 
No. 

NDB.Value 

Max.Yalue 

First Name " 



Middle Name (Optional) 





Last Name " 





Gender ” 





Date of Birth ” 





SSN* 


1 


^^B 

Home ZIP (strongly recommended) 


1 1 



HS Code (strongly recommended) 




^^B 

Criterion * 

First- Year GPA 




Cumulative GPA (if other than First- Year GPA) ” 


1 1 


1 — 

Retentio-n Indicator 




^^B 

Major area of study (when a^'ailable) 





Unh'ersit>'- As signed StLident ID (required) " 





Add"l Predictor 1 

Matrix 

1 1 



Add"! Predictor 2 

Need 

1 i 


1 I 

Add! Predictor 3 

Filed Fafsa 




Add"l Predictor 4 

Residency 




Add"l Predictor 5 

^TJE 

1 

1 

1 

Add'l Subgroup 1 

1 Abilit^^ rating in math 

From ACES 



Add"l Subgroup 2 

|Abilit>' rating in science 

From ACES | 

HS Avg * 


1 

1 

1 : 

Course 1 Label (abbredation) " 




^^B 

Course 2 Label (abbreviation) " 





1 _ _ _ ^ 
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Retrieving an ACES Study 


• ACES notifies the file submitter by e-mail when 
the study is completed 

• The document is password protected for 
confidentiality and is encrypted until downloaded 
at the institution 

• Results of ACES studies are confidential and only 
released to the institution that requested the study 

• Studies may also be mailed to an institution on a 
CD 
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Inside of the ACES Admission Vaiidity Report 


• Information on the most useful predictors of 
success at an institution 

• Optimal equations for predicting the success of 
future students 

• A list of the students at risk 
And... 

• A matched student-level data set for use in 
follow-up studies 
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ACES Admission Vaiidity Report 


ACES 




Admission Validity 
Report for Sample 
One University 

Mt* tn tiu« rappvt m« not f prwwattiw «f aRy inatltvtsen. Ml ar« 

BypotMcicM uM w*r« 9*e*r*c*4 tot cjm •»!« pvxp»M or craacuig cat* tMpi* v«pc«rt. 

Entering Class of FaD 2007 

Youl College Board ValiditY Report b designed to assist your 
ifistftuticn in vdidaling your adnusstort decisiorts. This report 
provides a nontechnical dscussion of important findmgs. 


ADMITTEX) CLASS EVALUATION SERVICE”* 
WWW.COLLEGEBOARD.COM 
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Evaluating Admission Measures 


Section 1: Evaluating individual admission measures 


Section 1: Evaluating individual admission measures 

Thb section summarizes the predictive strength of the individual admission measures in your study, first for the measures available 
for most of your students, and then for measures available for smaller groups of students. The second analysis may include results 
for predictors, such as SAT Subject Tests, that you did not explicitly choose to study but were present in your students' records. You 
may wish to consider the use of this additional infonriation for future admission decisions. 


See Section 2 for combinations of the individual measures, which are likely to provide more reliable and fairer information on your 
applicants. 


The tables below display the absolute value of correlations between each admission measure and First-Year GPA, the criterion you 
chose for this study. 



Other admission neesires available 


Strong Predictcro 


Moderate Predictors 


redicli'^ Strength Icorrolartion) 



Notes; 

• All individual measures have moderate to strong correlations with First-Year GPA except for the # Honors or AP courses measure. 
The measures showing moderate to strong correlations with First- Year GPA are good candidates for inclusion in the predicted 
First-Year GPA calculations in Section 2. 
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Section 2: Evaluating combined admission measures 


Section 2: Evaluating combined admission measures 

This section combines the admission measures that were evaluated individually in Section 1 of this report to find the best prediction 
of success. Combinations that are available for most of your students are presented first, followed by combinations that are 
available for smaller subgroups. 


Because combinations of predictors tend to be more reliable and allow stjdents to shojrv different strengths, it is important to 
consider all of the infbnnation available for a given student in making an admission decision. Appendix A presents the equations 
needed to combine the admission measures into a single predicted First-Year GPA. Several equations are given so thatyou can use 
as much of the infonnation provided to you by each student as possible. This section of your report gives you the infonriation you 
need to choose the best combination of predictors for each student. 

The tables below display the multiple oonelations between combinations of admission measures and the ineas'ire of success you 
chose for this study. The bars at the right of each table represent this predictive strength (multiple correlation) for each combination. 

The first table below presents SAT combinations. The first line of that table shows the multiple correlation for the predicted First- 
Year GPA using only SAT scores. 


SAT combinatioas 


sxr 

HS Rsit 

AM. 

nrsdclm 

N 

PlMictwsSt:ai|gh|mia(4tcEmlitiani | 

Oibiil 

XMi 

Witiig 

% 

31 

14 



97B 


IS 

t7 

70 

« 


2*1 


17 

t8 

20 

U 

' 

97{| 

05S 


SAT & SAT Subject Test coinbinatioas 


SJV 

SATSiliiKt 

IhI: 

HSflai 

AM 

N 

PlMid) w Stnn^ Indliplt i gmlitnvi 

Crtcd 

ftudng 

MMi 


Hqh- 

Nofiiig 

21 

27 

27 

a 



7S2 

[ JOSS 

t7 

ts 

K 

12 

3 


7S2 

OSD 

re 

20 

M 

11 

2S 

I 

7S2 

0S1 


Notes: 


♦ The multiple correlation calculated by using SAT Math, SAT Critical Reading, and SAT Writing was 0.44, which represents a 
strong conelation. The numbers in the boxes to the left of the bars show the relative contribution of each predictor (in percentage 
terms) for each prediction equation. SAT Critical Reading contributes 35 percent. SAT Math contributes 31 percent, and SAT 
Writing contributes 34 percent when using the SAT in predicting First- Year GPA 

• The second line of the SAT combinations table adds HS Rank to the SAT infonnation. Of the SAT and HS Rank. HS Rank makes 
the greatest contribution toward predicting Rrst-Year GPA After adding HS Rank, the multiple conelation increased from 0.44 to 
0.57. 
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Identifying Students at Risk 


Section 4: Using tlie pretUcted First- Year GPA (or current studenu to ideuttly students 
possibly at risk (or not completing Uielr degrees at Sample One Unleerslty 

Som saidHits Mtrad a FifM-Yeai Immi #im iiitpre4ctod by Ara prea^unn ai4weidi. kneaidi Iih jIkmh 4mm 
itidw to are al a ft toi iwtuooelaMig ilMwdupMi IhsailMifeoii can bauM^ to daniilv Mutants tnh»arapocel)lvai 
rnk (m loMsm Saitob Cbto Utovenib pr w to oiatuatMm 



bnportant pawn 

• A totol at lOa stodawtt vwara it eneb a l ashavatgaRnt-rbaf QfA t.jbatiwaai» lawn #Mn 4 m» piebctod bv fen pieebi BiH on 
dUfaciarsoo 

• TaypvoeUrfMiatoii»cnaMomMtane^(WUbMia%.iitop«atictotfim-V««CftK«baaAat(iadtoaa4^itutaM‘« 
lacofd an ilia ataceanc Mb i atumad u yau A hat ol IDs to sijjitots potsMy ai risk to rbapyim oai « vamtorng is psoaMad 
ai Apparaii C Saxa thto Int casaato sto Jaal MaceacaUana. yoa atop waat to Attach Assaatii C baloia ttolnbatoif 
Ibiartfoit 

• $totMb»wlx>aamada(»ra>-Vato<yAoHa>sllMn2J0aiai>atsiMionn V»aa(toC.aatlMM*tatantoaiaiaa(Mritana«>a4aa 
baaif at acadsaMC fk. 

• Thalhielsi9ast<kflafancasbaawa«npratficieilFtf»Yaa(GfAandaaaalFasfVaa(GFAafalaaadbalow bkaekeontoSM 

pedcaad antacaial Ft st-lf^ 6^. (toadema uAarnaaan b avaAabto to aack stotenTs tantei ant laitoetoaaiy I. as 

wel as Dwtottoi ii Ami Staton’s best language 

TtolalaMin^taMailae ie a t eMintoaMeflnaiinii i blaM kpsand n C 


1 
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in 
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im 
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XU 
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If 


To help you target 
retention efforts at 
Sample One University, 
the predicted First- Year 
GPA has been added to 
each student's 
record on the electronic 
file returned to you. A list 
of IDs for students 
possibly at risk for 
dropping out or 
transferring is provided 
in Appendix C. 
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Questions? 


Thank you! 

To access College Board research and reports: 
www.colleqeboard.org/research 

Feel free to email me with questions at 
eshaw@colleqeboard.org 

Researchers are encouraged to freely express their 
professional judgment. Therefore, points of view or 
opinions stated in College Board presentations do not 
necessarily represent official College Board position 
or policy. 
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